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Abstract. We explore properties of the value function and existence of optimal stopping times for functionals 
with discontinuities related to the boundary of an open (possibly unbounded) set O. The stopping horizon is 
either random, equal to the first exit from the set O, or fixed: finite or infinite. The payolf function is continuous 
with a possible jump at the boundary of O. Using a generalization of the penalty method we derive a numerical 
algorithm for approximation of the value function for general Feller-Markov processes and show existence of 
optimal or e-optimal stopping times. 
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The problem of optimal stopping of Markov processes has received continuous attention for last fourty 
years and produced diverse approaches for its solution. Foundations and general existence results can be 
found, e.g., in Bismut and Skalli |4i. El Karoui |8|, El Karoui et al. [9J, Fakeev [lOl, and Mertens lfT6l . 
From 1980s functional analytic methods gave way to a more explicit approach initiated by Bensoussan 
and Lions |)3|: value function was characterized as a solution to a variational inequality, which could be 
solved analytically or numerically. The main limitation of this method is the requirement of a particular 
differential form of the generator of the underlying Markov process. This paper belongs to another strand 
of literature which initially aimed at studying smoothness of the value function but also provides a different 
approach for the numerical approximation to the value function for a more general class of Markov pro- 
cesses (see Zabczyk ll22ll for a survey). These methods are not constrained by the form of generators and 
the development of the theory of PDEs. Specifically, we build on the penalty method introduced by Robin 
1(191 and generalized by Stettner and Zabczyk Il20l (see also 1211 ). which originates in ideas developed 
for partial differential equations but follows a purely probabilistic route. Of interest to numerical methods 
discussed in this paper is also a time-discretization technique explored by Mackevicius lfT4ll and further 
applied by Kushner and Dupuis [12] for numerical algorithms; see also Palczewski and Stettner ifTSl for its 
application to stopping of time-discontinuous functionals. 

We assume that the state of the world is described by a standard Markov process {X{t)) defined on a 
locally compact separable space E endowed with a metric p with respect to which every closed ball is 
compact (see the Appendix for the definition and properties of standard Markov processes). The Borel 
cr-algebra on E is denoted by £. The process {X(f)) satisfies the weak Feller property: 



where Co is the space of continuous bounded functions £ — > R vanishing in infinity, and P, is the transition 
semigroup of the process {X{t)), i.e., Pth{x) - E ' [h{X{t))] for any bounded measurable li : E ^ R. 

Let <9 c £ be an open set and tq = inf{f : X(t) iO} - the first exit time from O. We study maximization 
of several classes of functionals: 

(1) Stopping is allowed up to time tq- The payoff is described by a function G before tq and by a 
function // at t^: 



1. Introduction 



(1) 




+ l|T<Tol e "^Gis + t,X{t)) + l^r>To} e 



His + To,X(To))}, 
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where (s, x) 6 [0, oo) x £, o' > 0, r > and f,G,H : [0, oo) x £ — > R are continuous bounded 
functions. 

(2) Stopping is allowed up to time tq and the payoff is given by a function F : [0, ex?) x £ — > R which 
is continuous on [0, oo)xO and possibly discontinuous in the space variable on [0, oo)x(T: 



This, in particular, covers a complementary problem to ([U: with F continuous on [0, oo) x (5 and 
on [0, oo) X with a possible jump at the boundary [0, oo) x dO. 
(3) Stopping is unconstrained (infinite horizon, r = oo) or constrained by a constant T (finite horizon) 
with the following functional; 



where the payoff function F is continuous apart from a possible discontinuity on [0, oo) x dO. 

Optimal stopping problems of the first type were studied by Bensoussan and Lions |[3| for non-degenerate 
diffusion processes under assumptions that G < H and O is bounded with a smooth boundary dO. They 
used penalization techniques similar to ours but applied them on the level of variational inequalities. Gen- 
eraUzations were attempted by many authors in two directions: to extend the class of processes for which 
this approach applies and to relax assumptions on the functional; see, e.g., Menaldi ifTSl for the removal 
of restrictions on degeneracy of the diffusion, and Fleming, Soner Q for relaxation of many assumptions 
regarding the functional and the coefficients of the diffusion via viscosity solutions approach. Functionals 
of the third type recently gained a lot of attention. Lamberton ifTSl obtained continuity and variational 
characterization of the value function for stopping of one-dimensional diffusions with bounded and Borel- 
measurable payoff function F. His result, however, cannot be extended to multidimensional diffusions. 
Hassan and Ceci studied stopping of semi-continuous payoff functions F for diffusions and certain jump- 
diffusions in one dimension (fT], l2]l). They proved that value function for a functional with lower/upper 
semi-continuous function F is lower/upper semi-continuous. The existence of optimal stopping times was 
also shown but without an explicit construction. 

This paper complements existing theory in two aspects. Firstly, it provides results for a far larger family 
of Markov processes (in particular, in dimensions higher than 1) and enables numerical treatment of the 
value function. Secondly, it relaxes constraints on the region O, which can be unbounded and with non- 
smooth boundary. Our main assumption is that the mapping x i-> E '{l|ro</] h{Xt)] is continuous for any 
f > and a continuous bounded function h. This assumption is non-restrictive as we show in Section |5j 
It is usually satisfied by solutions to nondegenerate stochastic differential equations driven by Brownian 
or Levy noise. Consequently our results, based on probabilistic arguments, provide regularity of solutions 
to differential or integrodifferential variational inequalities, related to appropriate stopping problems, with 
various types of discontinuity. 

In our approach, the value function is approximated by a sequence of penalized value functions which 
are unique fixed points of contraction operators. These operators do not involve stopping or any other type 
of control, which makes them easier to compute numerically. Moreover, a discrete approximation of the 
state space can be used because we prove that the penalized functions are continuous. 

The remaining of the paper is organized in the following way. Section|2]introduces the penalty method 
for functionals of the first type. The following section explores the properties of the value function, in 
particular, its behaviour on the boundary dO. In Section|4]main results on optimal and e-optimal stopping 
and the convergence of penalized value functions are obtained. Sufficient conditions for the main assump- 
tion [(Al)] are formulated in Section |5j Functionals of the second type are studied in Section |6l Section |7] 
extends these results to functionals of the third type with infinite time horizon. A finite time horizon setting 
is studied in Section[8] Important properties of Feller processes are listed in the Appendix. 



(2) 




(3) 
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2. Penalty method 

We solve the stopping problem O]) using the penalty method introduced by Robin |fT9l and generalized 
by Stettner and Zabczyk ll20l . For yS > consider a penalized equation 

e'""[f{s + u,X{u)) + /3{G{s + u,X{u)) - v/(i + m,X(m))) ]du 

+ e-""'His + To,X(To))]. 

LEMMA 2.1. Assume that g and h are bounded functions and a > Q. For any bounded progressively 
measurable process {bit)), the following formulae 

(5) z{s, jc) = E ■'j " e-""g{s + u, x(u))du + e-''^°h(s + tq, x(to))}, 

g-<H,-/„ b(t)dt^g^^ _^_ ^(j^-)) + b{u)z{s + M, x(u))]du 

(6) + e-"''°-i«° '"^'^'"his + TO, x(to))} 

are equivalent in the following sense: z defined in (|5]l is a solution to and any solution to (|6]l is of the 
form (|5]l. 

Proof. We use similar arguments as in Lemma 1 of ETI . The only difference is that now we have tq 
instead of the deterministic time T - s. □ 

Using this lemma, in a similar way as in Proposition 1 of |21|, we show 
LEMMA 2.2. There is exactly one bounded measurable function w^ that satisfies 
Proof. By Lemma IZTI the penaUzed function vv^ can be equivalently written as 

v/{s, ;c) = E '[ " e-*"+''"'[/(i + u, X{u)) 

^'^^ + /3[g{s + u, X{u)) - v/(i + u, X{u))y + pw'^is + u, X{u))\du 

+ e-^"^P^'oH{s + To,X{To))]. 
Hence, vv^ is a fixed point of the operator T defined for measurable bounded functions (p as follows: 

T4>(s, x) = E '{ " e-<"+^'"[/(i + M, X{u)) 

+ I3[g{s + u, X{u)) - 4>{s + M, Xiu))y + /3(f>{s + u, Xiu))]du 

+ e-<"+^>^''//(i + to,X(to))]. 

This operator is a contraction on the space of measurable bounded functions for any /? > 0. Indeed, T<p 
is identically equal to H on [0, oo) x (9' , whereas the contraction property on [0, oo) xO follows from the 
estimate 

T(f>i-T<f>2 < -^1101 -02lU. 

a + p 

This impUes that is a unique fixed point of T. □ 

We make the following assumption 

(Al) The stopped semigroup P]°h{x) - E '^{l^i^jg^ h{X{t))} maps the space of continuous bounded func- 
tions into itself. 

The following three lemmas prove continuity results which, in particular, will be used to show that is 
continuous. 
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LEMMA 2.3. Under (Al),for a continuous bounded function h : [0, oo) x £ — > (-oo, oo) the mapping 

(s,x) i-> P]°h{s,x) E '"{l(,<^^)/!(i + t,X{_t))) 

is continuous. 

Proof. Let (i„, x„) — > (i, x). By Proposition lA. 1 I for a given e > there is a compact set /(T c £ such that 

P-'" {3„e[0,s+/+i]^(M) ^ < e. 
For n large enough, i.e., such that \s — s„\ < 1, we have 
IP^^his^, Xn) - P]°h{s, x)\ 

< \E■'■^{l^,^ro}hiSn+t,Xm-^'''•{h,<ro}Ks + t,Xm\ 

+ |e ""{1 + t,xm - E-^{i„<,„) his + t,xm\ 

< S\\h\\ + |E '"{l(,<ro| l{X(,)eK} Ks„ + t,X(t)) - h(s + f,X(0)}| 

+ |e i(r<r«) h{s + f, xm - E ■^■{i„<,„) h(s + f, xm\ 

= s\\h\\ +a„ +b„. 

The sequence a„ converges to by uniform continuity of /i is on [0, i + f + 1] x K. Assumption (Al) impUes 
b„ 0, which completes the proof. □ 



LEMMA 2.4. Under assumption (Al) the mapping 

' Jo 

is continuous for any function h e C([0, oo) x E) and y > 0. 
Proof. Fubini's theorem implies 

r-TQ /^OO 

E-*] + m,X(m))£/m} = e-T"ip{s,u,x)du, 

Jo Jo 

where <p{s,u,x) - E '*{lj„<TQ) + u,X{u)}. This function is continuous in {s,x) for any fixed m > by 

Lemma |23] Dominated convergence theorem concludes. □ 

LEMMA 2.5. Under (Al) for a > Q and a continuous bounded function h : [0, oo) x £ i— > (— oo, oo) the 
mapping 

is,x) E''[e-"^°his + To,XiTo))} 

is continuous. 

Proof. Assume first that 

e-'"'h{s + u,X{u))du] 

for a continuous bounded function h. Using this decomposition we write 

/-»oo 

H(s, x) = E "[e-"'''>h{s + TO, X(to))} = E ' { I e-''"h{s + u, Xiu))du] 

J TO 

XTO 
e-""h{s + u,X{u))du]. 

Hence, H is continuous by Lemma [Z4l 

By the weak Feller property of {X{t)) functions of the form (|8]l are dense in Co([0, oo) x E) (see Lemma 
3.L6 in |l6|). Hence, H is continuous for h in Co. The extension of this result to continuous bounded 
functions h uses Proposition lA. 1 1 in the appendix. Fix a compact set K c E and S > 0. For any T,e > 
there is a compact set L c £ such that 

F\X(t) i L for some t e [0, T]) < s, V x g K. 
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Define r{s, x) - e-p(' i')-('-('5+r)) /;(^^ ^-^^ where p{x, L) denotes the distance of x from the set L. Such r is in 
Co([0, oo) X £) and by preceding resuhs R{s,x) - E^l^e^"'^'^ r{s + tq,X(to))^ is continuous. By definition 
r = lion [0, S +T]x L.LetA- {X(t) i L for some t e [0, T]}. The distance of R and H is bounded in the 
following way: 

\\H(s, x) - R(s, x)\\ = E 'le-^^^ih - r)(s + tq, X(to))] 

= E-'{e-«^<"^^l,^<l {h - r){s + TO A r,X(To A T))} 
+ E^{1,A| e-'''°{h - r){s + To,X{To))] 
+ E-''{l,^<l {e-''^''(h - r)(s + to,X{to)) 

- e-"''"'^{h - r){s + TO A T,X(To A T)))} 
< + \\h - r\\E + e-"^2\\li - r\\. 

This imphes \\H{s, x) - R{s, x)\\ < 2\\h\\{E + le'"'^) for {s, x) e[0,S]x K. Since T and e are arbitrary this 
implies continuity of // on [0, 5 ] x K. Hence, H is continuous on its whole domain by the arbitrariness of 
S,K. a 

COROLLARY 2.6. Under (Al), the unique bounded solution wfi of ^ is continuous. 

Proof. Lemmas [2 .4| and |23] impl y that the operator T introduced in the proof of Lemma lZSl maps the space 
of continuous bounded functions into itself. Since this operator is a contraction on the space of bounded 
measurable functions it is a contraction on the space of continous bounded functions. This implies that wfi 
as a unique fixed point is continuous. □ 

To establish convergence of vv^ to w as — > oo we introduce two additional representations of vJ^ . 

LEMMA 2.7. Under assumption (Al ), the function has the following equivalent representation: 

(9) y^{s,x) = sup{y(i,x,T)-E-*{l,,<,^|e-"^(G-v/r(i + t,X(t)))}. 

r 

Proof. Markov property implies that for any stopping time cr the following equality is satisfied: 

e"""[/(i + u,X{u)) + /3{G{s + u,X{u)) - v/(i + u,X{u))) ]du 

+ e-"'<"'""^V(i + TO A cr,X(To A cr))}. 

This gives the lower bound: 

(10) v/(i,Ji:) > E-'{ j e-""f(s + u,X(u))du + e-"'<^o'''^V(i + to A cr,X(To A cr))}. 
Further, 

v/(i, Jc) > E e-""f{s + u, X{u))du 

(11) ^Jo 

+ 1|,<,^| e-'«^[G - (G - ^^fiT]{s + tr,X((r)) + 1,,>,^| e-^'oH^, + to,X(to))}, 
because vJ^ - H on and G - {G - < . Define a stopping time 

cr* = inf{« > : wfi{s + u,X{u)) < Gis + u,X{u))}. 
Due to the continuity of G and (see Corollarv l2.6l ) we have 

hcr'<To} y^{s + (r\X{cT*)) < 1|,.<,^, G{s + cr\X{cr*)). 
This implies that for cr* the inequalities in ( fTOl i and (fTTT l become equalities and (|9]l follows easily. □ 
LEMMA 2.8. The function has the following equivalent representation: 

wP{s,x) = sup E-1 e"'"'"/o + m,X(m)) + h{u)G{s + u,X{u)S\du 

beMii Jo 

e-"-o-£<'hmH{s + To,X{To)% 



(12) beMf, 



+ I 
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where Mp is the class of progressively measurable processes with values in [0,/?]. 
Proof. By Lemma IZTI the function w^ has the following equivalent formulation: 

v/{s, jc) = E ■'"j ^ " e-'"'-/o" + u, X{u)) 

+ p(g{s + u, X{u)) - w^{s + u, X(m))) + b(u)w^(s + u, x(u))^du 

+ e-"^°-^^° + TO, x{to))\ 

for any progressively measurable process h(t) with values in [0,yS]. Since h{t) < [iv^t have 

I3[g{s + u,X{u)) - v/(i + u,X{u))f + b(u)M^{s + u,X{u)) > b{u)G{s + u,X{u)), 
which implies 

\^{s,x) > E '^{ " e-'™-/o"''«'"[/(i + u,X{u)) + b{u)G{s + u,X{u))\du 

+ h{s + TO, XiTo))]. 

This is an equality for b{t) given by 

bit) = 1^' ^ ^' ^^^^^ - "^^^ ^' 
10, otherwise. 

Hence, formula ( fT2] l is proved. □ 

PROPOSITION 2.9. Under (Al), the functions w^(s, x) increase pointwise to w(s, x) as /3 ^ oo. 

Proof. Equation fT% implies that the functions H^(i, x) are increasing in /3. Hence the limit w°°(i, x) - 
limyj^co w^(s, x) exists. By (|9]l we have w^ < w and, therefore, w°° < w. To prove that w°° = w we first 
show that w°° > G. Let x e O and, for rj > 0, put Z7''(m) = l|„<,ji /?. Then by ( fT2] i we have 

M^is, jc) > E ■'"{ ^ " e-'"'-/o" ''"«'"/(^ + u, X{u))du 

e-("+fi^"/3Gis + u,X(u))du + e-''^''-fo'' ''''('^'"h{to,X{to))] 
= E^{(/) + (//) + (///)}. 

Letting j8 — » oo we can make (/) and (///) arbitrarily small and for sufficiently small 77 and large j3 the term 
(//) is arbitrarily close to G(s, x). Dominated Convergence Theorem implies w°°{s, x) > G{s, x). 
From (|9]l for any stopping time t we have 

w^(i, x) > J{s, X, t) - E -'"{1|^<^^, e-"'{G - v/)^(i + t,X{t))}. 

By letting ^ 00 we obtain 

(13) w°°{s,x)> J{s,x,t), 

because limp^ooiG - w^)'^{s, x) = for x e O. Since t is arbitrary we conclude that w'"(s, x) - w(s, x) for 
X e O. For x e E \ O we have H^(i, x) = x) = w(i, x). □ 

COROLLARY 2.10. Under (Al), the value function w is lowersemicontinuous. Moreover, ifw is contin- 
uous then wfi approaches w uniformly on compact sets. 



Proof. The semicontinuity of w follows from Corollarv 12.61 and Proposition 12.91 Dini's theorem implies 
uniform convergence on compact sets if w is continuous. □ 
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3. Properties OF THE VALUE FUNCTION w 
In this section we explore the properties of the value function, in particular, its behaviour on the boundary 

of a 

THEOREM 3.1. Under (Al ), for x e dO we have 

(14) lim w{s,y)^G\/ His,x). 

The proof of this theorem consists of several steps which are of interest on their own. They are formu- 
lated and proved as separate results below. 

It is clear that vv > G on <9 and w - H on the complement of O. It is therefore natural to expect a 
discontinuity at the boundary of <9 if G > H. The following proposition shows that this discontinuity is 
constrained to the minimum: the absolute value of the difference between G and H. 

PROPOSITION 3.2. Assume (Al ) and G > H. For any x e dO we have 

(15) Hm w(s,y) = G{s,x) 

and the convergence is uniform in s and xfrom compact sets. 

Proof. Since w{s,x) > G(s,x) for x e O and G is continuous we obtain that liminf,.^ v, ye(9>v(i,y) 
> G(s,x). In the remaining part of the proof we show that limsup^.^^ ^.^Qw(s,y) < G{s,x), which im- 
plies that the limit in (flSl l exists and equals G(s, x). 

Fix a compact set K c E, T > and £ > 0. First we make preparatory steps. By Proposition |AT| in the 
Appendix, there is a compact set L c £ such that 

(16) supP"(3 r e [0, r + 1] X(t) i L) < s. 

xeK 

The extension of the time interval by one unit to [0, T + 1] is required to allow the initial time s to be in 
[0, r] and leave time for the process {X(t)) to evolve. Notice that below 6 and rj are both bounded by 1. 
Let 6 € (0, 1) be such that for (s, x) e [0, T] x B(L, 6), y e L, \\x -y\\<6 and t € [0, 6] 

(17) \G{s,x)-G(s + t,y)\<s, 

Proposition I A. 3 1 implies that there is 77 > 0, which, for convenience, is bounded by 5 A e, such that 

(18) sup supPX^(0 i B{x, 6)) < s. 

xeL l<q 

Fix X 6 dO n K and s 6 [0, T]. For any y € (9 n /iT we have 
w{_s,y) - sup7(s,y, t) 

T 

< supE 'l e-""f{s + u,X(u))su + e""*^''^<'>G(5 + r A to,X{t A to)) 

T Jo 

< ¥>\to > 77)(— + IIGII) + r\To < i]Mf\\ + G(s,y)) 

a 

+ supE-^{l|,^<„ |e-"(-^-«>G(i + T A To,X{T A to)) - G(s,y)\}. 

T 

Consider the last term. For any stopping time t we have 

E-''{l(ro<,) |e-"<"^"<'*G(i + T A To,X(T A to)) - G(s, x)\} 

< ||G||(1 - e-"'') + E^\G(s + tAtoAt],X(tAtoA tj)) - G{s,x)\] 

< ar]\\G\\ + E^\G{s + t Atq Atj,X(t Atq A rj)) - G{s + rj, X{rf))\] 

+ Ey[\G{s + Ji,X{i^))-G{s,y)\\ 
- (/) + (//) + (///). 
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The first term is bounded by ae||G||. The estimate of the second term requires conditioning onX(TATo At/), 
the use of the strong Markov property and inequalities (fTST i, (fTSl i: 

E^\G{s + t a to a //,X(t a to a 77)) - G{s + t],X{t]))\} 
= Ey{E^^^^^<'^'''{|G(i + T A TO A T],X{0)) - G(s + t],X{t]-t Ato A t]))\]] 
<2\\G\\¥^3se[0,T]]Xis)m 

+ 2\\G\\ W{\/s G [0, 77] Xis) 6 L and X(t]) i B{X(t A to A rf), 6)] + e 
< 2\\G\\s + 2||G||£ + e = g(l + 4||G||). 

Term (///) is estimated similarly knowing that y is in L by assumption: (///) < e(l + 2||G||). Combining 
these estimates we obtain 

w{s,y) < + IIGII) + (1 - h,{y)M\f\\ + G{s,y)) + e(2 + (6 + a)\\G\\) 

a 

< G{s,y) + h,{y)i}^ + 2IIGII) + 77II/II + s{2 + (6 + a)||G||), 
a 

where h,j{y) - P'{to > rj). Assumption (Al) implies that is continuous on E. Clearly, h,j(x) = 0. Hence, 

Urn sup w{s,y) < G{s, x) + j]\\f\\ + s{2 + (6 + a)||G||) 

y-»x, yeO 

and the limit in the right-hand side is uniform in (s, x) e [0, T] x (dO n K). Since e > is arbitrary and 
77 < e this implies ( fTsT l. □ 

COROLLARY 3.3. Under (Al ), for any x e O we have 

(19) lim sup w(s, y) <GV H(s,x) 

Proof. Notice that 

nrATo 

w(s,y) < supE'j e-""f{s + u,X(u))du + e-"(^^"«'G y H(s + t A to,X{t A to))} 

T ' Jo 

and then continue as in the proof of Proposition l3.2l replacing G with G V H. □ 

The following proposition explores the impact of the value of the functional on the complement of O on 
the value function close to the boundary of O. 

PROPOSITION 3.4. Assume (Al ). For each e > 0, T > and a compact set K C E there is a compact 
set Kg <zO such that for x € K \ K^, s 6 [0, T] and > Owe have 

(20) v/(i, x) > H(s, x) - s. 

Proof. Fix e' > and choose 77 > such that ([TSIl-lfTsTl hold for the function H. By the definition of w^ we 
have 

M^(s,x) > -II/IIt; _ Mp-^{^^ > Tj}+E'{e-'''°H{s + To,X(To))}. 
a 

Splitting the last term depending on whether To is greater or smaller than 77 and doing analogous estimates 
as in the proof of Proposition l3.2l we obtain the following lower bound 

M^(s, X) > (1 - h,(x))H{s, X) - /7,(x)(M + _ e'(2 + 6\\H\\ + ||/||). 

a 

By arbitrariness of e' > and continuity of h^ we can choose /Tg such that ( |20l ) is satisfied. □ 



According to Proposition l3.4l Assumption (Al) guarantees the "migration" of H into O, i.e., the function 
H provides a lower bound for when x approaches dO. As w^ is the lower bound for w (see Proposition 
12.91 ) this property is shared by the value function w. In particular, when H > G the value function smoothly 
rises to the upper level H on the boundary of O. 
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Proof of Theorem \3.1\ From ( |20] i. letting first [} ^ oo and then e — > we obtain 

lim inf w{s, x) > H{s, x). 

Since G is continuous and w{s,y) > G(s,y) on O this extends to 

lim inf w{s, x) > G V H{s, x). 

Corollary [33] and the above inequality imply that the limit in (fT4l i exists and equals Gv H. □ 

4. Continuity of w and existence of optimal stopping times 
Let D denote the set of functions (p{s, x) admitting the following decomposition: 

e-""(pi{s + u,X(u))du + e-^'^ip^is + to,X(to))} 

for 1^1 , (/32 6 Co([0, 00) X £). 

LEMMA 4.1. The set D is a dense subset o/Co([0, 00) x E). 

Proof. It follows immediately from the proof of Lemma 4 in ||2T1 . □ 
LEMMA 4.2. IfG has decomposition with (pi - guifi = 82 ^ Co([0, 00) x E) then 
H^{s,x)-Gis,x) > - "-^ ~ - E ■'{e'<'^+^'^"||// - gall}. 

If, moreover, G < H then 

^(s,x)-G(s,x)>-^^l^. 

a + p 

Proof. Define w^{s, x) - w^(s, x) - G{s, x). Decomposition ( 1211 1 of G and representation (|4]i of vv^ imply 

vi^is,x) = E-^{ ° e-""[f - gi +yS(vv^)-](i + u,X{u))du + e-"^'\H - g2){s + to,X{to))]. 
By Lemma lZTI we have the following equivalent form of the above equation 

\^{s,x) = E-^"{ ^ " e'^"^'^'^"[f - gx + Piwl^r + I3\^]{s + u,X(u))du 

+ e-^''^P^'o^H-g2){s + To,X(To))}. 

Since (vv^)" + vv^ > 0, we obtain 

^^^(s,x) > E-'{ j| " - g^){s + u,X{u))du + e-^"^<''>^o(H - g2){s + to,X{to))\. 

This implies the first statement of the lemma. 

Due to decomposition (1211 1 we have g2{s, x) - G{s, x) for x i O. Together with the condition G < H 
this yields {H - g2)(s + tq, Xijo)) > 0. □ 

THEOREM 4.3. Assume (Al) and G < H. The value function w is continuous on E and an optimal 
stopping moment is given by 

(22) T*(s) = inf{f >0:w(s + t, X(t)) < G(s + t, X(t)) or X(t) ( O]. 

Proof. Functions are continuous (by Lemma 12. 2t . increasing in /3 and dominated by w. Therefore, 
it suffices to estimate the difference w - w^. For functions G with decomposition (ISTT i. Lemma l4~2l and 
equation (|9]l give 

fir \^ t \ 11/ -gill 
w{s, X) > w(s, X) . 

Since D is dense in Co([0, 00) x E) (Lemma [4.1| i we also obtain the continuity of w for G e Co([0, 00) x E). 

The extension of this result to continuous bounded G uses Proposition lA.ll in the appendix. Fix a 
compact set A' c £ and S > 0. For any T,s > there is a compact set L c £ such that 

¥'{X{t) i L for some t e [0, T]) < s, x e K. 
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Define G(s,x) - e p^"'^^ '■^^^"*G(i, jc), where p{x,L) denotes the distance of x from the set L. Let w 
be the value function corresponding to G. Since G e Co([0, oo) x E), preceding results imply that w is 
continuous. We also have \\w(s,x) - w(s,x)\\ < (e""^ + s)(}\f\\/a + ||G|| + \\H\\) forxeK and s e [0,5]. 
Since T and s are arbitrary this implies continuity of w on [0, 5 ] x K. By the arbitrariness of S,K the value 
function w is continuous on its whole domain. 
Define for e > 

(23) t\s) = inf {f >0:wis + t, X{t)) < G(s + t, X(f)) + e or X{t) i 0\ 
and 

(24) T«(i) = inf{f > : v/(5 + f, X{t)) < Gis + t, X(t)) or X(t) i 0\. 



Fix 5 > and T > 0. By Proposition ! A. 1 I for a given x e £ there is a compact set Kf, such that PM^a) ^ 1 
where = \X{t) e KsVt e [0, T]). From due to the Markov property of (X(t)), we obtain 

W'{s,x) < [P^'IA^l + e-"^P-MAa andT^(5) V t%s) > T}]i\\G\\ + \\H\\ + ^) 
+ E-'{l|A,| J e"'"'[/+yS(G-v/)+](i + M,X(M))^^M 

where cr*^y(s) = T/3(s) A t^(s) A tq A T. Notice that by the uniform convergence on compact subsets of 
vt^ to w we have P^{t8(5) AT < T^is) A T and Ag] — > as — > oo. Therefore letting — > oo we obtain by 
Dominated Convergence Theorem 

T llfll 

w(.,x)<(5 + e-"^)(||G|| + ||//|| + ^) 

a 

+ E*{l|A,| e-""f{s + u,Xiu))du 

+ 1|A„ e-^^^'^'^^^'^^wC* + tHs) a to a r,X(T^(^) A TO A D)}. 

Proposition ! A. 1 [ implies that (A^) form an increasing sequence of subsets when (5 — > and lim^^o P'^{A^) - 
1 . Therefore, letting 5 — » we get 

w(.,x)<e-''^(||G|| + ||i/|| + ^) 

a 



-IX 



e'""f{s + u,X{u))du 



+ e-«^'(">^^o''^w(i + T%s) A TO A T,X{t%s) a to a T))]. 
Now taking the limit r — » oo yields 

e"™/(i + u,X{u))du + e-"^^'^''''°w{s + t%s) A to,X{t%s) A to))]- 

Note that t^'{s) — > t*{s), as £ — > 0, and, further, by quasi-leftcontinuity of the process {X{t)) (see, e.g., 161) 
we also have X{t^{s)) — > X{t*{s)), P*-a.s.. Consequently letting e — > in ( l25T l and using the continuity of 
w give 

XT'{S)^T0 
e-""f{s + u,X{u))du + e-''''^'^''^°w{s + T*(i) A to,X{t''(s) A to))]. 

This implies that there is an optimal stopping time dominating t*{s). The optimality of t*{s) is now 
obvious. □ 

To prove the continuity of the value function w without the requirement of an upward jump we introduce 
the following assumptions: 

(A2) lim^^^o P''{t"c) < '?! = uniformly in x from compact subsets of O. 

(A3) {X{t)) is strongly Feller, i.e., the mapping x i-> E '{/i(X(f))) is continuous for any measurable bounded 
function h and f > 0. 
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Before we formulate Theorem |4.8| we prove three auxiliary results. Lemma |4~4l shows that under [(A3)| 
the time-state process semigroup maps time-continuous bounded functions into functions continuous in 
both parameters. Lemma |43] states that the weak Feller continuity of the process X{t) is sufficient for the 
continuity of the value function w in the time parameter s. Lemma l43] shows that |(A2)| follows from |(Al)| 

LEMMA 4.4. Under assumption \(A3)\ the mapping 

{s,x) ^ {F{s + h,X(h))] 

is continuous for h > and a bounded measurable function F, provided that the mapping s F(s, x) is 
continuous uniformly in x in compact subsets of E. 



Proof Fix a compact set A' c £ and T,e > 0. By Proposition I A. 1 I there is a compact set L c £ such that 
sup,e^ P^{X(/i) m <e. Hence for (s, x) e [0, T]xKv/e have 

\E-'{Fis + h,X(h))} - <l)(s,x)\ < \\F\\e, 

where (^{s,x) = E-'{l(A'(/,)eL| F{s + h,X(h))}. Let (i„,x„) {s,x) such that (s„,x„) e [0, T] x K for all «. 
By the continuity of F in i and by assumption |(A3)| for sufficiently large k, we have 

lim |<l)(5„,x„) - <l>(i,jr)| < \im\<l>{sk,x„) - 0(i,x„)| + lim |<l)(i, x„) - <l>(i,x)| = e + 0. 

By the arbitrariness of e this completes the proof. □ 

LEMMA 4.5. The mapping s i— » w{s, x) is continuous uniformly in x in compact subsets of E. 

Proof. Assume that s„ — > s and fix a compact set K c E. Since functions /, G and H are bounded and the 
discount rate a > 0, for any e > there is T > such that \ J(s„, x, t) - J{s„, x, t /\T)\ < e for all x e A" and 
n = 1, 2, By Proposition lA. 1 I there is a compact set L c £ such that for a\\ x e K and t <T we have 

EM,3,^,„„xw«L|{ e-'fis + u,Xiu))du 

Jo 

+ Ur<ro} e-"'Gis + t,X{t)) + Ur>ra} e-''^'>H{s + to,X{to))} < e. 
Uniform continuity of the functions /, G, and H on [0, T] x L yields 

XtAto 
e-"" {f{s + u, X(u)) - f{s„ + u, X{u))] du 

+ Ur<ro} e-"' {G{s + T, X{t)) - G{s„ + T, X{t))) 

{H{S + TO,X(to)) - H{S„ + TO,X(To))) } < B 

for T < r, a sufficiently large n and x € K. Consequently w{sn,x) — > w{s,x) as n — > oo uniformly in 
xe K. □ 

LEMMA 4.6. Assumption \(Al}\ implies \(A2)\ 

Proof. By Lemma |24l the function gy{x) - E '{e"'""") is continuous on O for any y > 0. By dominated 
convergence theorem gy{x) converges to when y — > oo and x e O. This convergence is monotone and, 
due to Dini's theorem, uniform on compact subsets of O. Chebyshev's theorem yields 

r{To < 77) = r{e-'o"^ >e-'}< egi/^(x). 

The right-hand side converges to 0, when rj ^ Q, uniformly on compact subsets of O, which completes the 
proof. □ 

REMARK 4.7. Assumption |(A2)| holds if the process {X{t)) satisfies the following continuity condition: 
(A2') foranye>0 

limP'l sup p(x,X{s)) > e) = 

'^0 je[0,(] 

uniformly in x from compact sets. 
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Such assumption is satisfied for a wide variety of Markov processes which are solutions to the stochas- 
tic differential equations with Levy noise with bounded coefficients. To prove this we simply use the 
Doob's maximal inequality to the martingale terms in the stochastic differential equation (see e.g. Theo- 
rem 1.3.8(iv) in lITTI ). 

Let for h > 

(26) wh{s,x) = E'[J^ e-""f{s + u,X{u))du + e-"''w{s + h,X{h))]. 

THEOREM 4.8. Under ^A2)\ and ^A3 )\ the function w is continuous on [0, oo) x O. Assume additionally 
(Al). The penalized functions are continuous and converge to w uniformly on compact subsets of 
[0, oo) X O. An e-optimal stopping time is given by 

t'{s) = infjf >0:w{s + t,X(t)) < G(i + t,X(t)) + s or X{t) i O}. 
Proof. By Lemmas l4.4l and l43] the function w/, is continuous in (s, x). Let t'^ = infjf > h : X{t) i O] and 

j'\s, jc, r) = E e-""f{s + u, X{u))du 

Jo 

+ e-^^Gis + t,X{t)) + e-<His + to,X(t^))}. 

By Theorem 3b of lfT6l applied to the Markov process consisting of a pair (s + t, X{t)) we have 



Consider an auxiliary value function 



wi,{s, x) - sup J (s, X, t). 

T>h 



Wh{s, x) - sup J{s, X, r). 



We have the following inequalities 

(27) \wh(s, x) - Whis, x)\ < C r[To < h] 
and 

(28) < w(s, x) - wi,(s, x) < sup {J{s, x, r) - J{s, x, t/,)} =: Ii,{s, x), 

T 

where ti, - t V h and C > 0. Assumption |(A2)| implies the difference |w;, - vv/,| converges to as /? ^0 
uniformly on compact subsets of [0, oo) x O. The proof of uniform convergence of //, is more involved. 
First notice 

Ihis, x) = sup {J(s, X, r) - J(s, X, h)} 

T<h 

< \\f\\h + 2{\\G\\ + \\H\\)F-'\to < h] + supE-'{G{s + t,X{t))} -E 'lGis + h,X(h))]. 

T<h 

It suffices to prove that as /i — > 

(29) sup E "{Gis + T, X{t))} - E '{Gis + h, X{h))} 

T<h 



uniformly in s, x in compact subsets of [0, oo) x O. By Proposition lA.2l we have 

lim E '{G(s + h, X(h))} = G(s, x) 

uniformly in s, x in compact subsets. Let v/,(s, x) - sup^^,, E '^{G{s + t, X(t))}. By weak Feller property this 
function is continuous (see, e.g., ifTSl Corollary 3.6] or |[22|). By dominated convergence theorem and the 
right-continuity of trajectories of X we have lim/,^o v/,(i, x) - G{s, x). Since this convergence is monotone 
and functions v/, and G are continuous Dini's theorem implies that v;, tends to G uniformly on compact 
sets. This completes the proof of ( |29] l. Consequently, w/,(i, x) converges to w(s, x) as /j — » uniformly in 
compact subsets of (s, x) € [0, oo) x (9 and w is continuous on [0, oo) x O. 

Assume (Al). By Corollarv 12 .6 [ functions are continuous. Dini's Theorem and Proposition |2.9| implv 
their uniform convergence to w on compact sets. In an identical way as in Theorem l4.3l we prove that t^(s) 
is well-defined and e-optimal (this last assertion follows directly from (|25]|). □ 
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Theorem 14 . 8 1 state s the continuity of w in [0, oo) x O. It is also clear that w is continuous on [0, oo) x O'^ 
because on this set w coincides with H. However, if there is a downward jump on the boundary of O 
{G{s, x) > H{s, x) for some x e dO) the function w has a discontinuity in this point. This follows from the 
observation that w > G on the set [0, oo)xO and w = // on [0, oo) x <9^. Therefore, the statement of the 
above theorem cannot be strengthened. This also implies that an optimal stopping time might not exist as 
the following example shows. 

EXAMPLE 4.9. Let £ = R and X{t) be a Brownian motion. Take O - (-oo, 1) and a < 1/2. It is easy to 
see that assumptions |(A1)|(A3)| are satisfied. Put G{s,x) = min(e*, e) and H{s,x) = fis,x) = 0. Notice 
that these functions do not depend on s, which impUes that the value function is also time-independent. We 
shall, therefore, skip s in the notation. 

LEMMA 4.10. In the setting of the example, 

(1) for X < \ and t >0 we have 

(30) l{t,x) E-^{e-'"l,x(o<iK''*"} = e^^--"^>--^Q>0-^^\ 

^ yft ' 

where O is the standard normal cumulative distribution function, 

(2) w{x) > l(t, x), forx< \ and t > 0, 

(3) w{x) > G{x),forx < 1. 

Sketch of the proof. The formula dSOl l can be calculated directly using the normality of X{t). To prove (2), 
define a sequence of stopping times t„ = inf{r : X{t) > 1 - 1 /«). Clearly, 

w(;c)>E-^{e-"(^"'^'>e^(^"^"} 

and 

lim ^'[e-^^-^-'^'^e^^'"'"^} > l(t,x). 

The proof of the last assertion rests on the observation that G(x) - 1(0, x) and ^'(0, x) > for x < 1. □ 

Assume that there exists an optimal stopping moment r* forsomex* < 1, i.e., w{x*) - E'''{e""^ 1(t'<to) G{X{t*))}. 
From the strong Markov property of the process X{t) we infer that G{X(t*)) - w{X{t*)), -a.s. on 
{t* < To)- Since w(x*) > e "' we have P ''(t* < tq) > 0. This is a contradiction with assertion (3) of Lemma 

KM 

REMARK 4.1L Penalty method offers a numerical procedure for solution of optimal stopping problems. 
Lemma IZTj provides an estimate of the error: \\w - w^\\ < \\{G - h^)^||. This error decreases as /3 increases: 
by Proposition |2.9| H^ forms a non-decreasing sequence of functions converging to w. Under [(AT)] functions 
are continuous (c.f. Corollarv l2.6b . Theorems l4.3l and [4.8l state assumptions under which w is continuous 
and is approximated by uniformly on compact sets. The continuity of w and w/} implies that state space 
discretization methods can be safely applied. Following Lemma 12.21 function w^ can be computed as a 
fixed point of a contraction operator T given by 

T^is,x) = E e"'"+^"'[/(i + u,X(u))+/3[g{s + u,X(u)) - ^{s + u,X{u)))^ 

+ I3(j){s + u,X{u))\du + e"*"+^'^o//(i + to,X{to))\ 

for a bounded measurable function (f). This operator can be implemented via PDF or Kushner-Dupuis 
space-time discretization approach (see lfT2ll ). The fixed point is approximated by an iterative procedure 
with an exponential decrease of the error (due to the contraction property of T). 

5. Sufficient conditions for (A1) 

Define for /; > 



Consider the following assumption: 
(A4) 



lim hJx) - 0. 

x^aO, xeO 
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This assumption ensures that when approaching the boundary of O the probability of crossing it in a short 
time converges to 1 . It is clearly satisfied (by Chebyshev inequality) whenever the mapping x E '{ro) is 
continuous. It can be viewed as a complementary assumption to |(A2)| We will show that |(A2)[|(A4)| imply 
(Al) and (Al) is sufficient for |(A4)| 

LEMMA 5.1. The function hjj is continuous on E under assumptions \(A2)\(A4)\ 

Proof. For 6 e (0, ?/) define r^ix) - 'E''{h^-sQ^{5))]. This function is continuous by |(A3)| The difference 
between rg and /j,, can be bounded in the following way: 

< rsix) - h,j(x) < ¥'{to < 6). 

Assumption |(A2)| states that the right-hand side of the above inequality converges to as 5 —> uniformly 
in X from compact subsets of O. Hence, h,j is continuous in O. It is identically zero on E \0. These two 
pieces fit continuously at the boundary of O because, due to |(A4)[ h,j{x) converges to as x approaches the 
boundary of O. □ 

The continuity of h,j implies uniformity of the limit in assumption |(A4)[ which is formalized in the 
following corollary. 

COROLLARY 5.2. Ifhij is continuous then for any compact set L c E and constants 7],e > there is an 
open set L^s such that L,^e c O and for each x e L \ L,,_£ we have P^Xro > rf) < b. 

PROPOSITION 5.3. Under ^A2%(A3)\ the mapping x i— > P]°h{x) is continuous on E\ dOfor any bounded 
measurable function h and f > 0. Ifadditionally ^A4)\ holds then P]° maps the space of bounded measurable 
functions into the space of continuous bounded functions and as a result condition (Al) is satisfied. 

Proof. Let h be a bounded measurable function. By the strong Feller property [(A3)[ for s < t, the mapping 
X i-> E '"^ jE^^'' {l[i-s<To) h{X{t - i))}} is continuous. Furthermore, 

E Mlu<.,) h(Xm = E- {l,,<,„| E^('* {1(,_,<,,| h(X(t - s))}] 

(31) 

= E ^ {e { 1 h{Xit - 5))}} - E { 1 E { 1 „_,<,^ j hiXit - s))}} . 

Therefore 

|EMl(,<r,| h(Xm {e^('> {1„_,<,^, h(X(t - s))]} I < ||/z|| ¥-'{To < s] < \\h\\ P'^to < 2^) ^ 



uniformly on compact subsets of C) as i — > by |(A2)| This shows the continuity of x i-> P]°h{x) for x eO. 
For X in £ \ (3 we clearly have E^{\[,<jg] h{X{t))] - 0. Now we prove continuous fit at the boundary of 
O. Assumption |(A4)| implies that |E'^"{1|,<to| h{X{t)))\ < P-* {f < tq) \\h\\ decreases to as x approaches the 
boundary. Hence, P]°h is continuous on E. □ 

Proposition 15 . 31 states that |( A2)|( A4)| are sufficient for Assumption (Al). The following lemma shows 
that |(Al)| implies |(A4)1 Recall that |(Al)| also implies [(A2)l see Lemma|431 

LEMMA 5.4. Under (Al) the function h^ is continuous on E, which, in particular, implies ^A4)\ 

Proof. Follows from the identity hri - P]f\, where 1 denotes a function identically equal 1. □ 

6. Stopping with discontinuities on 
In this section we explore a stopping problem with a more general payoff function F: 

J{s, X, t) = E -'j j e-""f{s + u, X{u))du + e-"^^'''"'^Fis + (t A tq), X(t A to))}, 

where /, F are measurable bounded functions that are continuous in s uniformly in x from compact sets 
and F is continuous on [0, oo) x O. In particular, F can be of the form 

F{s, x) = 1, ^.£(5, G{s, x) + 1 1^.^(3, H{s, x), 

where G, H are continuous bounded functions. This is a complementary problem to the one described in 
preceding sections; a discontinuity of the payoff manifests itself only when the process (X(f)) jumps to 
at the time to- For a continuous process iX{t)) the form of F outside of O is irrelevant and the problem 
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simplifies to stopping with a continuous payoff function G. However, if {Xit)) jumps at tq, the process 
migrates to the set O'^ and the value of the functional is given by H. 
Define a value function wis, x) - sup^ J{s, x, t). 

PROPOSITION 6.1. Under ^A2)\ and ^A3)\ the function w is continuous in O. 



Proof. As in Lemma 1431 using continuity of s {f{s,x),F{s,x)) uniform in x from compact sets we 
obtain that s i-> w(s, x) is continuous uniformly in x from compact sets. The rest of the proof follows 
similar lines as the proof of Theorem l4.8l □ 

Define a penalized equation (c.f. equation (HI): 

e'""[f + PiF - v/)^](i + u,Xiu))du + e-"^''F{s + to,X(to))}. 

As previously, this function is a fixed point of a contraction operator (see the proof of Lemma IZSl i. To 
establish the convergence of to w, we need the following technical lemma: 

LEMMA 6.2. Under \(AT]\ and \(A3]\ the mapping 

is,x)^E'{F{s + To,X{To))} 

is continuous in [0, oo) x O for any bounded measurable function F that is continuous in s uniformly in x 
from compact sets. 

Proof Lemma l4!4limplies that <i>h{.s,x) - E '{E'^^'''{F(i + h + to,X{to))]] is continuous for h > 0. Note 
that 

E'[F{S + T0,X(T0))] 

= E-^'{E^W {^(^ + + to,X{to))} + Uro<h) {F{s + To,X{To)) - E^C" {F(s + h + to,X{to))}) }. 
Hence 

\¥.'{F{s + To,X{To))]-^H(s,x)\<2\\F\\r{To<h}. 

The right-hand side converges to uniformly in x from compact subsets of O, as h 0, by virtue of 
Lemma l4~6l □ 

PROPOSITION 6.3. Under \(AT)\ and \(A3)\ 

(1) There is a unique measurable bounded solution to the above penalized equation. This function 
is continuous on [0, oo) x O. 

(2) Functions wfi are non-decreasing in /3. 



Proof. Existence of a unique bounded measurable solution follows from Lemma |Z2| Lemma l4^ and l6.2l 
imply the continuity of w^. Assertion (2) follows from Lemma |278] □ 

PROPOSITION 6.4. Under fAT]\ and ^A3 )\ the sequence of functions converges to w and this conver- 
gence is uniform on compact subsets of [0, oo) x O. 

Proof. Using continuity of F on [0, oo) x <9, in a similar way as in Lemma lZTj we obtain 
w^is, x) = sup [j{s, X, t) - E-''{1,^<^o| e-"\F - v/)^(i + r, X{t))]]. 

T 

Proceeding as in Proposition 12.91 we prove the pointwise convergence of to w. By Proposition 16.31 
functions are non-decreasing in which implies, by Dini's theorem, uniform convergence on compact 
sets of [0, oo) X (9. □ 
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7. Infinite time horizon 
Consider an optimal stopping problem with infinite horizon 

(32) w'^is, x) = sup 7°°(i, X, t), 

T 

where 

(33) J°°{s,x,t) = e-""f(s + u,X{u))du + e-"^F{s + t,X(t))]. 

Assume the process X(t) satisfies the strong Feller property [(A3)] a > and functions /, F are measurable 
bounded and continuous in s uniformly in x from compact sets. 
The penalized equation has the following form: for /3 > 

Xoo 
e'""[f + PiF - M^'"')+](i + u,X{u))du]. 

LEMMA 7.1. Assume \(A3]\ 

(1) There is a unique (in the space of measurable bounded functions) solution vv^'°° of the penalized 
equation ( I34l l and this solution is continuous. 

(2) The mapping s i— > w°°(i, x) is continuous uniformly in xfrom compact sets. 

(3) We have the following equivalent representation ofw^'°°: 

(35) v/'°°(i, x) = sup E -1 I e"'™"/o" + u, X{u)) + b{u)F{s + u, X(u))\duV 

heM/j Jo 

where Mp is the class of progressively measurable processes with values in [0,y8]. 
Proof. (1) Similarly as in Lemma [2?2l the function m^'°° is a fixed point of the operator 

g-(a+fi)"(^f + /30 + (i{F - (^)+)(i + u, X(uj)du\. 

This operator is a contraction on the space of measurable bounded functions, which implies that w^'°° is a 
unique fixed point of 7~°° on this space. Lemma l4!4l implies that 7"°° maps the space of measurable bounded 
functions into the space of continuous bounded functions. Hence, m^'°° is continuous. 

(2) The proof is similar to that of Lemma 14. 5 1 We use the continuity of 5 i-> (/(i, x), F(s, x)) uniform in 
X from compact sets. 

(3) This assertion follows immediately from Lemma 1278] with O - E. □ 



LEMMA 7.2. Assume there isJiQE such that for x e J{ 

XOO 
e'''"(p{_s + u,X{u))du\, 

where (f> : [0, oo) x ^ — > R is measurable and bounded. Then 

Xoo 
g-^i-H-/?/; l,.v,„.,., (^^ g [0, oo) X J[. 

Proof. First notice that for any bounded measurable function ^ the following representations are equivalent 

Xoo 
e-''"^{s + u,X{u))du], 

v{s,x) = E-*{ J e-™-/o" + u,X(u)) + b(u)v(s + u,X{u))\du] 

for any bounded progressively measurable process b{t) (compare to Lemma lTTI with O - E). 
Define 

Xoo 
e'""[f + P(y^'°°T - <P\{s + u,X(u))du], 
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where vi^'°° - vt^ °° - F. Notice that vv^'°° coincides with w^'°° on [0, oo) x Jl. Applying equivalence 
for f = / +yS(H^'°")- - 4> and b(u) = /31(X(„)e^) yields 

Since (n^-")" + h^-°° > on [0, oo) x we have 



>-ll/-0li: 



Xoo 



This completes the proof since h^ °° = vi^-°° - F on [0, oo) x y[. □ 

We impose the following assumptions on F: F(s, x) - G{s, x) for x e O and F(s, x) = H{s, x) for 
X e 0'^\dO, where G and H are bounded continuous functions. Notice that F can be arbitrary on [0, oo)xdO 
as long as it is continuous in s uniformly in x from compact sets. In particular, F can be equal to G or // 

on 5a 

LEMMA 7.3. Under assumption (A3) we have 

(38) v^'°°(i, x) > sup X, r) - E '[e-'^iF - w'^'°°f{s + r, X{t))]], 

T 

and if F > G V H on 80, i.e., F is upper semicontinuous, then \v^'°" has the following equivalent represen- 
tation: 

(39) v/'°°(5, x) = sup [r{s, jc, r) - E '{e-"\F - m^-°°)^{s + r, X{t))]\. 
Proof. We follow the proof of Lemma IZTl For any stopping time r we have 

v/'°°{s, x) = E-'{ ^ e-""{f + I3{F - v/'°°)+)(i + u,X{u))du + e-"'V'"(i + t,X{t)]. 

Since > F - (F - h^'"')+ we obtain dllll. Let tr = inf{M : w^'°°(i + u,X{u)) < F{s + u,X{u))}. On 
the set {cr < oo) the upper semicontinuity of F and the continuity of w^'°° implies w^''^{s + o-,X{cr)) < 
F{s + cr,X(o-)). Combining this with a trivial result on the set {cr = oo) yields 

v/'°°(5, x) = J{s, x,cr), E '{e-'"'{F - v/'°°)+(i + a, X{a-))] = 0. 

This, together with ( [38] l, implies representation ( [39] l. □ 

In what follows we shall need the following two assumptions: 
(A5) For any x € dO we have 

Um 0-^ = P-'-a.s., and lim cr^ = P*-a.s., 

where 

o-£ = inf{M > : X{u) eE\{Ou F,)}, 
= inf{M > : X{u) eE\{(7u FJ), 
and Fg is the e-neighbourhood of dO: 

Te - {x e E : inf \\x - y\\ < e). 

yedO 

(A6) ¥^X{T) edO} = for any xeEandT > 0. 

REMARK 1 .A. Assumption |(A5)| is satisfied whenever each point of 5(9 is regular for (9 and £'\(C)u5(9) (see, 
e.g., Blumenthal and Getoor ||5l for a definition and properties of regular points). Indeed, 15] Proposition 
10.4] implies that To > limg^o cr^, where Tq is the first hitting time of O, i.e. 

To = inf{f > : X{t) € O}. 
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Take x e dO. Its regularity means that Tq - Q P^-a.s.. Therefore, limg^o = P'-a.s.. The convergence 
of CTg to can be proved in an analogous way. 

Assumption |( A6)| is satisfied whenever Markov process (X(f)) has a density at time T with respect to a 
measure which puts zero weight on the set dO. 

THEOREM 7.5. Assume ^AT]\ and fA3l 

(1) w°° is continuous on [0, oo) x (£ \ dO). 

(2) w°°'°°(i, x) :— limy3_>co w^'°°{s, x) is lower semicontinuous (l.s.c.) with values in R U {oo). 

(3) iff is l.s.c, then w"' is l.s.c. andw°°'°° > w°°. 

(4) if assumptions ^A5)\(A6)\ are satisfied and F < G V H on [0, oo) x dO, then m^'°° converges to w°°, 
as p ^ CO, uniformly on compact subsets o/[0, oo) x (£ \ dO). 

Proof. Let w^{s, x) - sup^>,, J°°{s, x, t). Theorem 3b of 1161 implies 

jc) = E ^{e""'' w°°(i + h, X{h)) + J e^^/Ci + u, X{u))du]. 

By Lemmas 14.41 and 17.11 the function w'j^ is continuous for each h > 0. Under |(A2)| which follows by 
Remarkl4r7lfrom |(A2')| in the same way as in Theorem l4.8l we prove that — » w"', as h — » 0, uniformly 
on compact subsets of [0, oo) x (£ \ dO). This implies assertion (1). 

Assertion (2) follows from Lemma ITTI Indeed, m/'°° is non-decreasing in /? and continuous for each /3. 
Hence, the limit w°° °° is well defined and lower semicontinuous. 

Define w°°{s,x) - sup^^o x, t). Notice that w'^(s,x) t w"'(s,x) as h -> 0, which implies w°° is 
l.s.c. |fT6l Theorem 3b] implies w°°{s,x) - max{w°"{s, x), F{s, x)}. Hence, if F is l.s.c, then the mapping 
w°°(s, x) is l.s.c. as maximum of two l.s.c. functions. Applying ( [35l l with b{u) - y61 ji,<;,) and a sufficiently 
small h yields > F. Letting y8 ^ oo in 01 and using w°°'°° > F we obtain > w°°. This 

completes the proof of assertion (3). 

Last assertion is the most demanding. We assume first that F - G V H on dO. We will relax this 
assumption later By Lemma [73] we obtain w°°'°° < w°° . The proof of the opposite inequality is divided 
into several steps. Assertion (4) will then follow from Dini's theorem. 

Step 1. Assume G = Rag and H - Rah, where the functions g, h : [0, oo) x £ — > R are continuous 
bounded and the resolvent is defined in Lemma lTSl It is sufficent to consider g,h e Co([0, oo) x E), but 
it does not simplify the reasoning in any way. 

Lemmas |7.2| and r73] imply the following bound: 

V/'"(5,X) > sup [r(s,X,T) - E^'{l|XWeO| 11/ - g\\<t^o(^{T))} 

T 

- E^{l,XWeO'A.50) 11/ - hWcf^^g^iXiT))] 

-Ell, 



{x(j)^ao] M, 

a 'J 



ll/-gl|v||/-/»lh i 



where, for an open set A c £, we define 

By dominated convergence theorem, lim^j^co (f>^{x) - for x € A. Taking a limit as /? — > oo yields 
(40) w°°''" > sup [r(s, X, t) - MP'"{X(r) € dO}], 

T 

whereM = (||/-g||V||/-/i||)/a. 

Step 2. We will show that the supremum in ( l40l i can be restricted to stopping times satisfying ¥^X{t) € 
80} = 0. Fix a stopping time t and define for e > 

T + (o-s o Or), if X{t) i 80, 
= i r + (cr^ o Or), if X{t) € 80, F{s + r, X{t)) = H{s + t, X{t)), 
T + (cr^ o Or), if Xir) e 80, F{s + t, X(t)) = G{s + t, X{t)), 

where ds = inf{M > : X{u) i Fg) and cre,(T^ are defined in assumption |(A5)| The stopping time Tg 
might attain the value oo, in which case the functional J is also well defined due to discounting. Notice 
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the difference between ds and cTg (cr^): the former is the first exit time from the e-neighbourhood Fg of 
dO, whereas the latter is the first exit time from <9 U Fg {(T U Fg, resp.). The stopping time Tg equals t for 
appropriately small s if X{t) i dO. Otherwise, i.e., when X{t) e 80, it follows from assumption |(A5)| that 
limg^o Te — > T P'-a.s. \f F - H ?X the time t then X{Te) e (if it is finite) and by the continuity of H we 
obtain 

Hm e-"''^H{Te, X(ts)) = e-"^H(T, X(t)) P-'-a.s. 
We proceed similarly when F = G at the time t and get 

Hm e-"^^F{T^, X(tJ) = e-°^F{T, X{t)) P^-a.s. 

Dominated convergence theorem implies 

lim J'^{s, X, Tg) = J^(s, X, r). 

We also have W{X(tb) e 80} ^ for each e > 0. Hence, 

X, t) = lirn [7°°(s, jc, t^) - Mf '{X(T^) e 80}] < sup [7°°(s, jc, f) - Mf '{X(f) e 80}]. 

Combining this result with ( l40t yields 

/CX) / \ J- 00,00/ \ 
(S, X,T) < W (S, X), 

which, due to arbitrariness of t, gives the required inequality < 

Step 3. Using standard methods we extend above result to continuous bounded G and // in a similar 
way as in Theorem |4.3l 

We relax now the assumption F = G V H on 80. Let F be as in the statement of the theorem and 



Fis,x) = 



\F(s, x), X i 80, 

\(G V W){s, x), X e 80. 

Denote by w^-°° and vv°° the value functions corresponding to F . Fubini's theorem and assumption |(A6)| 
imply that 

E '{ e-™l,z(„)eaO|flfM} = 0. 

Using this equality we obtain 

v/'°°(i,x) 

+ u,X{u)) + UximdO}PiF - + u,X(u)) 

+ hx(u)edO}/^iF - w^n^s + u,X(u))]du] 

Xoo 
+ u,X(u)) + Ux(.0i3O)KF - w^n\s + u,X(u)) 

+ '^{X{u)edO\l3{F - w^-°°Y{s + u,X{u))]du^ 

'[f{s + u,X{u))+l3{F - wP-'^Yis + u,X{u))]du], 



where the second equality follows from the fact that F coincides with F onE\ 80 and the third term under 
the integral integrates to zero. Since h^'°° is a unique solution of the penalized equation ( l34l i with function 
F (see Lemma lTTt we conclude that m^'°° = h^'°°. 

Denote by J°° the functional 7°° with the function F. Fix any stopping time t and define for e > 

T + o-s o Or, if X{j) i 80, 
Ts^It + cr,,oer, if X{t) € 80, F{s + t, X{t)) = His + t, X{t)), 
T + cr^o Bj, if X{t) e 80, F{s + t, X(t)) = G(s + t, X{t)). 

Similarly, as in Step 2 we obtain 

\imr(s,X,Ts) = /°°(i,JC,T), 
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which impUes vv°° < Opposite inequality is obvious SiS F > F. 

In the first part of the proof of assertion (4) m^'°° was shown to converge to vv°° uniformly on compact sets 
in [0, oo) X (£ \ dO). Since w^''^ coincides with m^ °° and vv"" coincides with this uniform convergence 
holds for w'^ ™ and □ 

REMARK 1 .6. The complexity of the proof of assertion (4) in Theorem l7.5l is caused by the incompatibility 
of the continuity conditions that one has to impose on the function F. On the one hand, we need to prove 
that w°°'°" > F, which requires that F is lower semicontinuous. On the other hand, the inequality w^-"" < w°° 
is true under the condition that F is upper semicontinuous (see Lemma 1731 1. 

8. Finite time horizon 

Methods from previous section can be applied to optimal stopping of the following functional: 

J'^{s,x,t) = E-'l J e-''"f(s + u,X{u))du + e-«(^^*^-*»F((i + t) A T,X{t A {T - s)))}, 

where c > 0, the function / is measurable bounded and continuous in s uniformly in x from compact sets, 
and F has the following form: F{s, x) - G(s, x) for x eO and F{s, x) = H{s, x) for x e (T\ dO for bounded 
continuous functions G and H. This functional is a finite time horizon version of the functional J°'{s, x, t). 
Denote the value function by w^(s, x) - sup^ J^{s, x, t). Notice that can be equivalently written as 

w{s,T)- sup J°°{s,x,f). 

T<T-s 

To enable numerical approximations of this value function we introduce a penalized equation: 

(41) v/-^(s,x) = E "^{ J e-™[/ + p{F - vv^-^)^](i + u,X{u))du + e-"^'^-'^F{T,X{T - s))]. 

LEMMA 8.1. Assume ^A3]\ 

(1) There is a unique measurable bounded solution to ( I41l l and this solution is continuous on [0, T)xE. 
Under Assumption \(A2 ')\ the continuity extends to (JO, T) X E) U {T X (E \ dO)). 

(2) This solution has an equivalent representation: 

v/'^(i, jc) = sup E-^j I e"™"/o + u,X{u)) + b{u)F{s + u,X(u))\du 

(42) beM/i Jo 

+ e-'''-'^-''>-f»''''^'^'"F{T,X(T - s))}, 

where A/g is the class of progressively measurable processes with values in [0,y6]. 

(3) w^'^ is increasing in /3. 

Proof. Similarly as in Lemma IZ21 we show that there is a unique bounded measurable solution to (l4Tb . 
Lemma |4j4] implies that this solution is continuous on [0, T) x E. The continuity onT x{E \ 80) is more 
delicate. Fix x eO and a sequence (s„, x„) c [0, T}xO converging to (T, x). We have 

|v/'^(^„,x„) - ^^'\T,x)\ < (ll/ll +I3\\{F - v/'^)1|)(r - s„) + 2F||P^"(to <T - s„) 

+ \¥.-'"{e-''^^-'"^G{T,X{T - s„))] - G{T,x)\. 

The first term vanishes as n — > oo. Assumption |(A2')| implies that the second term converges to zero. The 
convergence of the third term follows from the continuity of the mapping (see Proposition |A.2| ): 

{s,x,h) ^ E-'{e-''''G(s + h,X(h))}. 

Convergence to x e \ dO can be proved in an analogous way. 

Representation ( l42b is obtained in an analogous way as in the proof of Lemma l2.8l Assertion (3) follows 
immediately from ( |42] |. □ 

THEOREM 8.2. Assume ^ATWATj\ 

(1) The function w^ is continuous on [0, T] X (£ \ dO). 

(2) w°°'^(s, x) :— lim^_,co w^'^(s, x) is lower semicontinuous on [0, T) X E with values in R U {oo). 

(3) ifF is l.s.c, then is lower semicontinuous on [0, T)x E and w""'^ > w^. 



STOPPING OF FUNCTIONALS WITH DISCONTINUITY AT THE BOUNDARY OF AN OPEN SET 



21 



Assume further \(A5j\ \(A6)\ 

(4) If F < G V H on [0, T] X dO then w^'^ converges to as j3 ^ oo uniformly on compact subsets 
of[0,T]x(E\dO). 

(5) The mapping (s, oo) 9 r i— > w^(s, x) is continuous for fixed s and x e E \ dO. 

Proof. Similarly as in the proof of Theorem l7.5l we show that is continuous on [0, T)x{E \ dO). The 
extension of the continuity to T x (£ \ dO) follows an analogous route as in the proof of Lemma [8T| Fix 
X eO and a sequence {s„, x„) c [0, T]xO converging to {T, x). Then 



I T 

\w 



{s„, x„) - w'^iT, x)\ < \\f\\(T - s„) + 2\\F\\ r"(To <T- s„) 



+ sup E'"{e-"^G{s„+T,X(T))]-G{T,x)\. 

't<{T^s„) ' 

The first term vanishes as n — > oo. Assumption |( A2 ' )] implies that the second term converges to zero. The 
convergence of the third term follows from the continuity of the mapping (see ITSl Corollary 3.6]) 

(i, X, h) ^ sup E ^{e-"^G(i + T, X{t))]. 

Convergence to x e \ dO can be proved in a similar way. 

Assertions (2)-(3) are proved in a similar way as in Theorem l7.5l 

Assertion (4) is trivial for s - T since w^'^{T, x) = w^{T, x) = F{T, x). In the following we will address 
the case s < T. In a similar way as in the proof of Theorem 17.51 we show that it suffices to prove both 
assertions for F = G V H on dO. Under this condition, as in Lemma |731 we show that the function n/'^ 
has the following equivalent representation: 

(43) v/'^(i,x) = sup{y^(i,jc,r) - E-*{e-"'^^'''^-"»(F - v/'^)+((i + t) A T,X{t A (T - s)))}]. 

T 

This implies that w°°-^ :- Um^^oo h^'^ < w^. The proof of the opposite inequality requires similar but 
slightly more delicate argument as in the proof of Theorem l7.5l 
Step 1. Assume 

Gis, x)^E e-''"g{s + u, X(u))du + e-^^^-'^GiT, X(T))], 

H{s, x)^E J e"™/j(s + u, X{u))du + e-"'^'^-'^H(T, X{T))], 

for continuous bounded functions gji. Combining arguments from proofs of Lemmas 14.21 and [7.21 we 
obtain 



(G - M^'^Tis, X) < 11/ - g\\cfj-\x) + \\F - G|k-(«+W^-", 
{H - v/'^)+(i, X) < 11/ - hW^.-^Jx) + \\F - 



where, for an open set A e E, 

//(x) = E^{ e-'«'-^/o"i|«"-^i'"c/M}. 
Above estimates and identity ( |43] | imply the following bound: 

w>^'^is,x)> sup [j^{s,x,t) - E''[l^r=T-s)(F - w'^'^y(T,X(T))] 

T<T-S 

-E'[Ur<T-s) Uxir)eO} 11/ - ^ll^'^"''^'\^(r))} 

- E-'[Ur<T-s) Ux(r)eC>\aOi 11/ " h\\4/'^\J^^'\X(T))} 

- Jti \^l{T<T-s} ^{X{T)eaO} j 

-E^{l,,<r-.HIG-^^IK'""'''^'"*"^"^^ 
Taking the limit as j6 ^ oo and recalling that w^'^{T, x) = F(T, x) yield 
(44) w°°'^(i, x) > sup [j'^is, X, t) - MY\X{t) e dO}], 



T<T-s 
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whereM = (||/-^||V||/-/i||)/a. 

Step 2. Fix 6 > Q. For any T > 0, s e [0, T) and a stopping time t < T - s define for e > 

T + (o-e o 6»r) A 6, if X{t) i dO, 

T + (o-g o 6^) A 5, if X(t) g 5C), F(s + T, X{t)) = H(s + T, X{t)), 
T + (o-^ o 61^) A 5, if X(t) 6 50, F(s + T, X(t)) = G(i + T, X(t)), 

where o-g = inf{M > : X{u) i Fg) and cr^, cr^ are defined in assumption |(A5)| Contrary to the proof of 
Theorem |73] the difference - t is bounded by S. This, however, does not affect the Hmits as e ^ 0. If 
F - H ?A the time r then X{Te) e (9' for appropriately small e and by continuity of H we obtain 

Hm H{t^, X{t^)) = H{t, X(t)) P-'-a.s. 

We proceed similarly when F = G at the time r and get 

Hm F(Te, X(t£)) = F(t, X(t)) P-'-a.s. 

Dominated convergence theorem implies 

lim J°°{s, X, Te) = J°°{s, X, t), and lim PM^(Te) e a<9) = 0. 

Recalling that < T - i + 5 we obtain 

r{s, X, Te) - MPMX(tJ e dO] < sup x, r) - MP{X(t) e dO]]. 

T<T-s+S 

As £ ^ the left-hand side converges to J^{s, x, r). The arbitrariness of r implies 
(45) w^(i, x) < sup jc, t) - M¥{X(t) e dO}]. 

T<T-s+6 

Step 3. Combining formulas (l44l i and ( l45T l we get for any 6 > Q 

w°°'^(s, jc) > w^^-'Ci, x), (i, ,y) € [0, r - 5] X £. 
Take a stopping time t < T - s and define Tg = t A (T - s - 6). We have 
hm sup X, Tg) - X, t)| 

< lim supE -*{|e""^'>F(i, x, ts) - e-^Tis, x, t)\] 

< lim supE-'^{l,^<7--i) \e-''^'F{s, x, Tg) - e'^^Fis, x, t)|) 

(5^0 

+ lim supE '{l(^=7-_,, x(T)edO] |e""^'/^(i, x, Tg) - e'"^F{s, x, t)|} 
+ lim supE '{l(^=7-_^, X{T)<tdO\ \e^"'''F{s, x, Tg) - e'"''F{s, x, t)\] 
= (l) + (2) + (3). 

Limit (1) equals from dominated convergence theorem. By quasi-left continuity of the process X{f) and 
dominated convergence theorem limit (3) is as well. Term (2) is dominated by 2||F||P'{X(r - s) e dO], 
which by |(A6)| is equal to 0. Hence, 

lim v/^^{s, x) - v/{s, x). 

(5— >0 

This completes the proof of both assertions in the case when G and H can be written in resolvent forms 
(assertion (4) follows from Dini's theorem and assertion (1)). 

Step 4. Using standard methods we extend above result to continuous bounded G and // in a similar 
way as in Theorem |4.3l 

Step 5. We relax the assumption that F = G V // on [0, T] x dO as in the proof of Theorem |7.5l □ 
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Appendix Appendix A Properties of weak Feller processes 

A Markov process defined on a locally compact separable space is called standard (see 0, p. 104, or 
E Definition 9.2]) if 

(1) it is a strong Markov process, 

(2) it is cadlag and quasi-left-continuous, 

(3) the filtration is complete and right-continuous. 

Let {X{t)) be a cadlag Markov process defined on a locally compact separable space (E, S) endowed 
with a metric p with respect to which every closed ball is compact. Assume that this process satisfies the 
weak FeUer property: 

PiCqCCo, 

where Co is the space of continuous bounded functions £ — > R vanishing in infinity, and Pih{x) - 
E ' [h{X{t))] for any bounded measurable h : E ^ R. Right continuity of {X{t)) and Theorem Tl, Chapter 
XIII in ifTTll implies that the semigroup P, satisfies the following uniform continuity property: 

lim P,/ = / in Co, V/eCo. 

/->0+ 

Theorem 3.1 (p. 104) in ||6l implies that there exists a standard Markov process on the state space E with 
the semigroup P,. In fact, it follows from the proof of the aforementioned theorem that the process {X{t)) 
satisfies the conditions of a standard process if its filtration is complete. The filtration of {X{t)) can be 
completed without changes to other properties of the process due to Proposition I A. 21 below and Theorem 
3.3 and Subsection 3.6 in ||6l. 
Let 

(46) rr(x, R) = {3,e[0,T] p{x, X{s)) > R] . 
PROPOSITION A.l. fin Proposition 2. 1]) For any compact set K Q E 

(47) supy7-(jc,/?) ^ 

as R ^ oo. 

PROPOSITION A.2. fUl Corollary 2.2]j 

i) PtC C C, where C is the space of continuous bounded functions £ — > R ( the Feller property). 

ii) lim,^o Ptf(x) — fix) uniformly on compact subsets of E for f e C. 

PROPOSITION A.3. Theorem 3.7]j For any compact set K c E and any e,6 > there is ho > 
such that 

sup sup P'iXih) ( B{x, 5)} < E. 

0<h<ho xeK 
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